Papers with W4A8 quantization
Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are proficient in natural language processing tasks, but their deployment is limited by extensive parameter sizes and computational demands. |
| Approach: | They propose a method to enhance computational efficiency in large language models by 4-bit weight and 8-bit activation quantization. |
| Outcome: | The proposed techniques significantly boost task accuracies to levels comparable with full-precision models. |